Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add METIS headers #146

Merged
merged 12 commits into from
Jan 17, 2024
Merged

Add METIS headers #146

merged 12 commits into from
Jan 17, 2024

Conversation

hugobuddel
Copy link
Collaborator

@hugobuddel hugobuddel commented Nov 27, 2023

This adds a preliminary set of METIS headers to the METIS simulations.

@hugobuddel hugobuddel marked this pull request as draft November 27, 2023 20:03
@hugobuddel hugobuddel marked this pull request as ready for review December 1, 2023 11:58
@hugobuddel
Copy link
Collaborator Author

Soo, while these extra keywords work, the breaks the ad-hoc way METIS keywords were added before. As a result, METIS/docs/example_notebooks/LSS-YSO_model_simulation.ipynb is now broken:
https://github.com/AstarVienna/irdb/actions/runs/7059688144/job/19217764513

KeyError: "Keyword 'SLIT' not found."

These are the keyword that are produced by dev_master for lss_l mode. Primary:

SIMPLE  =                    T / conforms to FITS standard                      
BITPIX  =                    8 / array data type                                
NAXIS   =                    0 / number of array dimensions                     
EXTEND  =                    T                                                  
EXPTIME =                    1                                                  
HIERARCH detector_readout_mode = 'auto    '                                     
DATE    = '2023-12-01T14:33:13'                                                 
ORIGIN  = 'Scopesim 0.6.1'                                                      
INSTRUME= 'METIS   '                                                            
INSTMODE= 'lss_l   '                                                            
TELESCOP= 'ELT     '                                                            
LOCATION= 'Armazones'                                                           
SOURCE  = 'Table   '                                                            

and extension:

XTENSION= 'IMAGE   '           / Image extension                                
BITPIX  =                  -64 / array data type                                
NAXIS   =                    2                                                  
NAXIS1  =                 2048                                                  
NAXIS2  =                 2048                                                  
PCOUNT  =                    0 / number of parameters                           
GCOUNT  =                    1 / number of groups                               
WCSAXESD=                    2                                                  
CRPIX1D =              1024.5                                                   
CRPIX2D =              1024.5                                                   
CDELT1D =               0.018                                                   
CDELT2D =               0.018                                                   
CUNIT1D = 'mm      '                                                            
CUNIT2D = 'mm      '                                                            
CTYPE1D = 'LINEAR  '                                                            
CTYPE2D = 'LINEAR  '                                                            
CRVAL1D =                 0.0                                                   
CRVAL2D =                 0.0                                                   
LATPOLED=                90.0                                                   
MJDREF  =                 0.0                                                   
ID      = '1       '                                                            
X_CEN   =                  0.0                                                  
Y_CEN   =                  0.0                                                  
X_SIZE  = '2048    '                                                            
Y_SIZE  = '2048    '                                                            
HIERARCH pixel_size = 0.018                                                     
ANGLE   =                  0.0                                                  
GAIN    =                  1.0                                                  
EXPTIME =               3600.0 / [s]                                            
HIERARCH detector_readout_mode = 'auto    '                                     
DIT     =   0.3333333333333333 / [s]                                            
NDIT    =                    3                                                  
BUNIT   = 'e       '           / per EXPTIME                                    
PIXSCALE=              0.00547 / [arcsec]                                       
CTYPE1  = 'LINEAR  '                                                            
CTYPE2  = 'WAVE    '                                                            
CRPIX1  =               1024.0                                                  
CRPIX2  =               1024.0                                                  
CRVAL1  =                  0.0                                                  
CRVAL2  =             3.55E-06                                                  
CDELT1  =              0.00547                                                  
CDELT2  =            -6.35E-10                                                  
CUNIT1  = 'arcsec  '                                                            
CUNIT2  = 'm       '                                                            
DETECTOR= 'HAWAII2RG'                                                           
DET_MODE= 'fast    '           / detector readout mode                          
MINDIT  =                 0.04 / [s]                                            
FULLWELL=             100000.0 / [s]                                            
RON     =                   70 / [e]                                            
DARK    =                 0.05 / [e/s]                                          
ATMOSPHE= 'Skycalc '           / atmosphere model                               
LOCATION= 'Armazones'                                                           
AIRMASS =                  1.2                                                  
TEMPERAT=                    7 / [degC]                                         
HUMIDITY=                  0.1                                                  
PRESSURE=                0.755 / [hPa]                                          
PWV     =                  2.5 / precipitable water vapour                      
SURFACE1= 'LIST_ELT_combined.tbl' / telescope_reflection                        
SURFACE2= 'LIST_METIS_mirrors_cfo.dat' / common_fore_optics                     
ADC     = 'const_90'                                                            
SLIT    = 'C-38_1  '           / slit_wheel                                     
PUPTRANS=                  1.0 / cold stop, pupil transmission                  
SURFACE3= 'LIST_METIS_mirrors_img_lm.dat' / img_lm_optics                       
FILTER1 = 'L_spec  '           / filter_wheel                                   
FILTER2 = 'open    '           / nd_filter_wheel                                
QE      = 'QE_detector_H2RG_METIS.dat' / quantum_efficiency                     
PSF     = 'PSF_LM_9mag_06seeing.fits' / point spread function                   
SPECTRAC= 'TRACE_LSS_L_sci.fits' / spectral trace definition                    
WCSAXES =                    2                                                  
DETLIN  = 'FPA_linearity_HxRG.dat'                                              

And these are created with this branch, primary:

SIMPLE  =                    T / conforms to FITS standard                      
BITPIX  =                    8 / array data type                                
NAXIS   =                    0 / number of array dimensions                     
EXTEND  =                    T                                                  
EXPTIME =                    1                                                  
HIERARCH detector_readout_mode = 'auto    '                                     
MJD-OBS = '2027-05-04T02:30:00'                                                 
RA      =                  0.0                                                  
DEC     =                  0.0                                                  
ORIGIN  = 'SCOPESIM'                                                            
TELESCOP= 'ELT     '                                                            
INSTRUME= 'METIS   '                                                            
AIRMASS =                  1.2                                                  
HIERARCH ESO DPR CATG = 'SCIENCE '                                              
HIERARCH ESO DPR TYPE = 'OBJECT  '                                              
HIERARCH ESO DPR TECH = 'IMAGE,LM'                                              
HIERARCH ESO OBS ID = 42                                                        
HIERARCH ESO OBS NAME = 'TODO    '                                              
HIERARCH ESO OBS START = 'TODO    '                                             
HIERARCH ESO OBS TPLNO = 1                                                      
HIERARCH ESO TPL ID = 'METIS_img_lm_obs_GenericOffset'                          
HIERARCH ESO TPL NAME = 'METIS_img_lm_obs_GenericOffset'                        
HIERARCH ESO TPL EXPNO = 0                                                      
HIERARCH ESO TPL START = '2027-05-04T02:30:00'                                  
HIERARCH ESO DET DIT = 0.3333333333333333                                       
HIERARCH ESO DET NDIT = 3                                                       
HIERARCH ESO DET NCORRS NAME = 'TODO    '                                       
HIERARCH ESO DET CUBE MODE = 'F       '                                         
HIERARCH ESO INS DROT POSANG = 0                                                
HIERARCH ESO DRS FILTER = 'L_spec  '                                            
HIERARCH ESO DRS NDFILTER = 'open    '                                          
HIERARCH WISE ATMO LOCATION = 'Armazones'                                       
HIERARCH WISE ATMO ALTITUDE = 3060                                              
HIERARCH WISE ATMO LONGITUD = -70.1918                                          
HIERARCH WISE ATMO LATITUDE = -24.5899                                          
HIERARCH WISE ATMO TEMPERAT = 7                                                 
HIERARCH WISE ATMO HUMIDITY = 0.1                                               
HIERARCH WISE ATMO PRESSURE = 0.755                                             
HIERARCH WISE ATMO PWV = 2.5                                                    
HIERARCH WISE DET IMAGE_PL = 0                                                  
HIERARCH WISE DET TEMPERAT = -230                                               
HIERARCH WISE DET DAR VALUE = 0.05                                              
HIERARCH WISE DET SHO STATE = T                                                 
HIERARCH WISE DET QEC FILENAME = 'QE_detector_H2RG_METIS.dat'                   
HIERARCH WISE DET LIN FILENAME = '!DET.linearity.file_name'                     
HIERARCH WISE TEL NAME = 'ELT     '                                             
HIERARCH WISE TEL TELESCOP = 'ELT     '                                         
HIERARCH WISE TEL TER FILENAME = 'TER_ELT_6_mirror_field_track.dat'             
HIERARCH WISE TEL REF FILENAME = 'LIST_ELT_combined.tbl'                        
HIERARCH WISE OCS PXSCALE = 0.00547                                             
HIERARCH WISE OBS TPLNO = 0                                                     
HIERARCH WISE OBS INSTRUME = 'METIS   '                                         
HIERARCH WISE OBS NAME = 'MyScienceTest'                                        
HIERARCH WISE OBS RA = 0.0                                                      
HIERARCH WISE OBS DEC = 0.0                                                     

and secondary:

XTENSION= 'IMAGE   '           / Image extension                                
BITPIX  =                  -64 / array data type                                
NAXIS   =                    2                                                  
NAXIS1  =                 2048                                                  
NAXIS2  =                 2048                                                  
PCOUNT  =                    0 / number of parameters                           
GCOUNT  =                    1 / number of groups                               
WCSAXESD=                    2                                                  
CRPIX1D =              1024.5                                                   
CRPIX2D =              1024.5                                                   
CDELT1D =               0.018                                                   
CDELT2D =               0.018                                                   
CUNIT1D = 'mm      '                                                            
CUNIT2D = 'mm      '                                                            
CTYPE1D = 'LINEAR  '                                                            
CTYPE2D = 'LINEAR  '                                                            
CRVAL1D =                 0.0                                                   
CRVAL2D =                 0.0                                                   
LATPOLED=                90.0                                                   
MJDREF  =                 0.0                                                   
ID      = '1       '                                                            
X_CEN   =                  0.0                                                  
Y_CEN   =                  0.0                                                  
X_SIZE  = '2048    '                                                            
Y_SIZE  = '2048    '                                                            
HIERARCH pixel_size = 0.018                                                     
ANGLE   =                  0.0                                                  
GAIN    =                  1.0                                                  
EXPTIME =                    1                                                  
HIERARCH detector_readout_mode = 'auto    '                                     
INHERIT = 'T       '                                                            
EXTNAME = 'DET1.DATA'                                                           

On the one hand it seems trivial to 'fix' the notebook, but maybe it would be better to see whether it is possible to add the missing headers back in. Or similar headers, as the old ones are not ESO compliant.

@carmeloarci
Copy link
Contributor

carmeloarci commented Dec 1, 2023 via email

@hugobuddel hugobuddel requested review from oczoske and removed request for oczoske December 7, 2023 08:36
@teutoburg teutoburg added the instrument-specific Limited to a certain IRDB instrument package label Dec 7, 2023
@hugobuddel
Copy link
Collaborator Author

Copied from slack conversation with Francesco

FYI, I figured out why the WCS info disappears. At some point we added effects specifically for setting header keywords. These are supposed to replace the ad-hoc way we wrote headers in the write_header() function. There is an if-statement that skips write_header() when there are FITS header effects defined. And while write_header() included the WCS coordinates, there is no FITS header effect to add them. I'm not sure where the D coordinates are added. I'll see whether we can replace all functionality of write_header() with FITS effects, so we can remove that function entirely.

https://github.com/AstarVienna/ScopeSim/blob/36df1767c2b8143a199d085cdf1af181660c058a/scopesim/optics/optical_train.py#L314-L319

@hugobuddel
Copy link
Collaborator Author

We (HB, JK, FH, KL) discussed the headers in [https://github.com/AstarVienna/ScopeSim/blob/f9e10426d283d3188a9094782b9e5a5b50f7f8ff/scopesim/optics/optical_train.py#L334](write_header()] in optical_train.py.

Ultimately it would perhaps be best to put all headers in FitsKeywords Effects, which is then included by default. But at least for now it is easier to keep write_header(), clean it up so it contains only headers we always want (mostly WCS), and call it always before calling the FitsKeyword effects.

The plan:

  • Keep the WCS headers, but copy those from the image plane if possible, otherwise use astropy to construct them. The logic about the SpectralTraceList WCS should at some point be moved to earlier in the simulation so that no special logic should be necessary here, but for now we can keep it.
  • Anything ScopeSim specific, e.g. filenames and things we cannot know with the real data (like dark current), should be removed. The existing fits header effects should cover those, e.g. SimulationConfigFitsKeywords and EffectsMetaKeywords. It is up to the user / instrument to include those, but maybe we can make them a default.
  • Source related keyword will be replaced by SourceDescriptionFitsKeywords.
  • For the other keywords we will decide whether they are generic enough to be included, or instrument specific, and should be moved to the IRDB.

I realize now that we should check whether the existing keyword effects are suitable enough as a default. As in, we don't want to break too many workflows of people; maybe for some information, like filter selection, we should do something sensible as default.

We decided we can merge this PR without doing all of the above, because we have a plan, and merging this PR will only affect METIS.

@hugobuddel
Copy link
Collaborator Author

The headers now have WCS coordinates, so merging this in as described above.

@hugobuddel hugobuddel merged commit e0ba6f9 into dev_master Jan 17, 2024
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
instrument-specific Limited to a certain IRDB instrument package
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants